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ABSTRACT: This article considers the role of research in disentangling an 
increasingly complex relationship between literacy policy and practice as it is 
emerging in different local and national contexts. What are the tools and 
methodologies that have been used to track this relationship over time? 
Where should they best focus attention now? In answering these questions this 
paper will consider three different kinds of research perspectives and starting 
points for enquiry: 1. Policy evaluation. The use of a range of quantitative 
research tools to feed policy decision-making by tracking the impact on pupil 
performance of different kinds of pedagogic or policy change (Organisation 
for Economic Co-operation and Development [OECD], 2010). 2. Co¬ 
construction and policy translation. This has for some time been a central 
preoccupation in policy sociology, which has used small-scale and context 
specific research to test the limits to the control over complex social fields that 
policy exercises from afar (Ball, 1994). Agentic re-framings of policy at the 
local level stand as evidence for the potential to challenge, mitigate or re¬ 
order such impositions. 3. Ethnographies of policy time and space. 
Ethnographic research tools have long been used to document community 
literacy practices, and in training their lens on the classroom have sought to 
focus on the potential dissonance between community and schooled practices. 
It is rarer to find such research tools deployed to explore the broader policy 
landscape. In the light of debate within the field, part of the purpose of this 
article is to examine how ethnographic research tools might be refined to 
study how policy from afar reshapes literacy practices in the here and now. 
(Brandt and Clinton, 2002). 
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LITERACY POLICY, RESEARCH AND PRACTICE: NEW 
RELATIONSHIPS FOR NEW TIMES 

In recent times, and in many different jurisdictions, policy-makers have begun 
to exert much more direct influence over educational practice than once would 
have been the case. Literacy policy commonly stands centre stage in this 
process. Policy-makers’ interests in literacy are most often voiced in terms of 
efficacy (How well are we doing compared to others?); equity of outcomes (Is 
our system producing equal levels of attainment for all?); levels of skill (Are 
the levels of skill high enough to compete in the new global market?) and 
include choice of teaching method (Which methods mobilised in which order 
secure the best results?). These concerns resonate in wider public discourse. 
Few education systems have been found to measure up to this kind of close 
scrutiny. When this produces demand for change, policy-makers by and large 
set the terms under which this will happen. 
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Such concerns are not restricted to a single country. Rather there has been a steady 
convergence internationally on a similar set of criteria for comparing system 
performance and on a common set of tools for engineering change. Rizvi and Lingard, 
(2010), following Appadurai (1990), describe the combination of discursive and 
material resources which now largely shape education policy as a new “social 
imaginary” that has crowded out other ways of thinking: 

Over the past two decades the neoliberal social imaginary...has transformed thinking 
in almost all policy arenas including education....Neoliberalism has steered education 
policy priorities towards a particular curriculum architecture with an emphasis on the 
skills and dispositions needed for participation in the global knowledge economy, 
modes of governance that have highlighted principles of privatization and choice, and 
an audit culture that stresses performance contracts and various...regimes of testing 
and accountability which have thinned out the purposes, pedagogies and potential of 
education. (Rizvi & Lingard, 2010, p. 197) 

These changed modes of thinking have begun to erode earlier commonsense 
understandings of the point and purpose of education. In particular they have 
weakened the assumption that teachers and students are primarily engaged in building 
common cultures and identities as they develop shared knowledge. Instead schools are 
now encouraged to trade in skills and qualifications that will act as passports for their 
pupils to a future somewhere else. Education becomes a means to another end, rather 
than an end in itself. From a sociological perspective, this creates new conditions in 
which the individual student orients to the pedagogic discourse of the school and the 
school itself blends the instructional and regulative aspects of its pedagogy (Bernstein, 
1996; Moss, 2004). 


LITERACY POLICY REMADE: COMPARING PERFORMANCE 
OUTCOMES USING QUANTITATIVE MEASURES. 

One of the key ingredients in this new landscape has been the increasing use of 
quantitative indicators to capture different aspects of system perfonnance and drive 
decision-making in the policy sphere. Perhaps this is best exemplified by the role the 
OECD’s Programme for International Student Assessment (PISA) has played in 
instigating school reform. PISA uses a range of test instruments, sampling and survey 
techniques to compare students’ competence in reading, maths and science in 
participating countries. The data are collected, analysed and published on a three 
yearly cycle. The analysis makes strong links between the measures of student 
performance, structural elements of national education systems, the potential for 
education system reform and the contribution such reforms might make to economic 
performance. This feeds policy-makers’ rationale for taking education seriously. It 
also keeps the focus tied to the desired outcomes at system level, overlooking the 
more localised and immediate contexts and micro patterns of social interactions in 
which teaching and learning actually take place. 

In their list of FAQs, the OECD identify the value of the exercise in these terms: 

Does PISA tell participants how to ran their schools? 
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No. The data collected by PISA shows the successes of some participants' schools 
and the challenges being faced in other countries/economies. It allows countries and 
economies to compare best practices and to further develop their own improvements, 
ones appropriate for their school systems. 

http://www.oecd.org/document/53/0,3746,en 32252351 32235731 38262901 1 1 

I l,00.html 


From the point of view of the OECD, the data do their own work in the public 
domain, as countries review the findings and react accordingly. The assumption is that 
characteristics of the school systems in those countries that do best could usefully be 
borrowed by those countries doing less well and in this way all systems will converge 
on the same levels of performance (Steiner-Khamsi, 2004). Repeating the exercise at 
regular intervals - data collection, analysis, publication - produces new points of 
comparison, in relation to what has changed between time points. The analysis itself 
thereby becomes more nuanced. The work the datasets do rests with how these points 
of comparison are mobilised and understood. 

At its crudest, the analysis produces a rank ordering of countries, showing those with 
the strongest to weakest outcomes from their education system in terms of student 
performance. The results are often reported like this in national newspapers. But they 
also set policy-makers and politicians the task of explaining why their particular 
system is where it is and whether it needs to change. The need to explain and to 
change intensifies not so much according to the ranking per se, but according to 
where individual countries are in relation to their own education policy cycle, and 
how the data fit with, contrast with or even undermine current policy thinking and the 
investments in education it has led to. 

The politics to all this matters. In England, for instance, the PISA data have not 
strongly shaped the discussion on system reform. They have been used more simply 
to reinforce or counterpoint directions of travel politicians have already committed to, 
whether the data have shown the system doing well or pointed to weaknesses. In large 
part this is because England already collects and publishes its own system 
performance data. Its National Pupil Database (NPD) holds extensive information on 
individual pupils and continuously tracks their progress through an education system 
that tests frequently and throughout their school careers. In this context the NPD data 
already play a similar function to PISA data elsewhere, by fostering fine-grained 
comparative analysis of system performance, in this case conducted and then 
published at the level of the school. By contrast, Denmark has historically relied far 
less on examination and test data within its school system. The PISA data led it to 
consider whether it should use such data more (OECD, 2004). 

As the analysis of this kind of quantitative data has become more nuanced, so have 
the responses. For example, the table below breaks down the percentage of the sample 
performing at different Levels on the PISA test instruments for reading. This makes it 
possible to compare the distribution in student performance scores across Levels as 
well as the mean score for any one country. This can be used to identify longer tails of 
underachievement produced in one system as against another. (The data below are 
extracted from a larger table reported in PISA 2006 (OECD, 2006)). 
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LI 

LI 

L2 

L3 

L4 

L5 

OECD 

% 

% 

% 

% 

% 

% 

Australia 

3.8 

9.6 

21.0 

30.1 

24.9 

10.6 

Austria 

8.4 

13.1 

22.0 

26.2 

21.3 

9.0 

Canada 

3.4 

7.6 

18.0 

29.4 

27.2 

14.5 

Finland 

0.8 

4.0 

15.5 

31.2 

31.8 

16.7 

Ireland 

3.2 

9.0 

20.9 

30.2 

25.1 

11.7 

Korea 

1.4 

4.3 

12.5 

27.2 

32.7 

21.7 

N. Z. 

4.7 

9.9 

18.7 

26.4 

24.5 

15.9 

Spain 

8.7 

17.0 

30.2 

29.7 

12.6 

1.8 

U. K. 

6.8 

12.2 

22.7 

28.7 

20.5 

9.0 

OECD total 

8.9 

14.2 

23.1 

26.6 

19.2 

8.1 

OECD av 

7.4 

12.7 

22.7 

27.8 

20.7 

8.6 


Table 1. Percentage of sample performing at different Levels of reading (PISA) 


This level of analysis became a key element in the public discourse on literacy 
attainment in New Zealand, a country which came out very well on a “one number”, 
single mean score. On the latter indicator, the Ministry of Education in New Zealand 
commented: “Of the 572 countries participating in PISA 2006, the mean reading 
literacy performance of only three countries was significantly higher than New 
Zealand, two countries were similar, and the other 50 countries were significantly 
lower” (Telford & Caygill, 2007). 

Analysis of the distribution of student performance across Levels, however, led to a 
different set of questions which mapped onto prior concerns about the relative 
attainment of different sections of the population. 

In the international study of literacy achievement conducted in 1991 by the 
International Association for the Evaluation of Educational Achievement (IEA), New 
Zealand had the largest spread of scores of any of the participating countries (Elley, 
1992). In the most recent study by the IEA (Mullis, Martin, Gonzalez & Kennedy, 
2003), the standard deviations of achievement in reading ranged from a low of 57 for 
the Netherlands to a high of 106 for Belize. The standard deviation of New Zealand’s 
reading scores was 93. Similarly, results from the latest Programme for International 
Student Assessment (PISA) study (OECD, 2004) indicated that the distribution of 
New Zealand’s reading scores was larger than all but a few of the 41 participating 
countries. In each of these studies the majority of poor readers were from low-income 
backgrounds with an over representation of Maori students. (Tunmer, Chapman & 
Prochnow, 2006, pp. 183-4) 

This became the starting point for public debate over whether New Zealand should 
move away from its commitment to “real books” pedagogies in the early years 
towards phonics as the primary means of teaching reading. Tunmer and colleagues 
used their analysis of the PISA data to advocate phonics as a more effective method 
of teaching reading to the target populations who were doing less well. In the end this 
suggestion was not adopted (Soler & Openshaw, 2007). 
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Points of comparison in the quantitative data are not fixed, therefore, any more than 
the actions they engender. Rather such data lend themselves to multiple 
interpretations and responses in different national settings (Grek 2009; McGaw, 
2008). Despite what policy-makers might wish, part of the work they do is precisely 
to pose questions rather than give definitive answers: 

Analyses like these cannot offer definitive evidence about the causes of the variations 
or about specific actions to improve national performances. They can, however, 
suggest issues worth further exploration by other means. (McGaw, 2008, p. 226) 

Yet because of the way the statistical range in the data is interpreted, they also 
produce a general sense that most countries could do better in terms of equipping their 
citizens with the appropriate literacy skills. In and of themselves they offer little 
guidance on precisely how this could be done. The dilemma that policy-makers face is 
therefore two-fold. On the one hand keeping control over the political agenda so that 
any dissatisfaction with system outcomes can be safely managed; and on the other 
putting in place suitable remedies that can address the current state of affairs. 


WHEN POLICY SETS THE TERMS: FIXING THE FOCUS FOR LITERACY 
RESEARCH 

Using quantitative data to monitor literacy performance in this way has become 
commonplace, whether at national, district or school level. It fits modern modes of 
governance in education, facilitating “steering at a distance” (Kickert, 2005) by those 
who take political responsibility for educational outcomes, even though they operate 
at some distance from the schools and classrooms they seek to reform (Grek, 2009; 
Moss, 2009). In the policy domain itself expectations have been high that the 
problems can be fixed. If some systems produce better results than others, then this 
acts as a guarantee that any system could function at the same level, with the right 
mix of policy features. The notion that school systems express complex trade-offs in 
tenns of the relationships between school structures, school curricula, social 
geographies and the potential distribution of roles and functionings in the wider 
economy gets pushed to one side. 

Instead the research community is positioned as the source of the possible answers to 
a policy problem that others have defined in their own terms. This often amounts to 
being asked to fix the system’s results rather than look more broadly at a range of 
wider social factors that might need to change. Indeed it is often not within the scope 
of the literacy research community to pursue the latter course even if they wished to 
do so. On the contrary the very specialisation within research communities that leads 
to the development of expertise in particular areas that might be relevant to raising 
literacy standards - phonic instruction, for instance - may precisely preclude those 
same communities from knowing much about key elements in carrying out system 
reform -strategies for getting buy-in to a particular programme in different settings, 
for example. Yet such an alternative range of skills may be crucial to working at the 
scale policy-makers require. All of this is to draw attention to the fact that the answer 
to the question “how can we substantially increase literacy levels?” is unlikely to rest 
with substituting one method of teaching reading with another, though choice of 
methods and their sequencing may well play a part alongside other elements: the 
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general quality of instruction, models of professional development, or different ways 
of acknowledging and responding to social segregation within the student body, to 
name but a few. School effectiveness and improvement researchers have been 
amongst the first to draw attention to this complex mix of features and begin to find 
ways to orchestrate them in the process of trying out refonn (Fullan, 1993; Barber, 
1996; Levin, 2010). 

Refonn programmes do not exist as simple recipes that can achieve their ends by 
moving an intact knowledge base from here to there. More than ten years of attempts 
at large-scale reform have demonstrated this fairly conclusively (Linn, 2000; Earl, 
Watson & Katz, 2003). However, the logic of the policy field often drives policy¬ 
makers to act as if this were precisely the case. From their point of view, discussion 
of the data in the public domain continues to lead to broad-brush assessments of 
comparative failure or success which need channelling in different directions in the 
restless search for improvement. The desire to achieve better outcomes, more than 
the state of play in the available knowledge base or the conditions required to enable 
research to develop, drives the timelines and speed at which reform is implemented 
(Moss, 2009). 

Look across different jurisdictions and one can see this play out in various ways in 
relation to literacy policy, through the sequential championing of different methods of 
literacy instruction (for example, mandatory phonics programmes in California or 
Success for All in New York (Cobum & Stein, 2010)); more rarely through the co¬ 
option of lead academics into the policy domain (Luke, 2002); or through the 
introduction of intensive, centrally-driven programmes of professional development, 
that have largely bypassed existing research communities to reach deep into the social 
organisation of schooling (Levin, 2010). Often the “lease” such programmes have on 
the policy space is provisional. A change of government or disappointing results can 
all lead to one programme being dropped and another being adopted (Earl et al., 
2003). 


UNDERSTANDING THE DIFFERENCE BETWEEN FIELDS: LITERACY 
AND ETHNOGRAPHIES OF PRACTICE 

In the account I have given above, I have brought out the differences between how 
policy-makers and researchers orientate to the business of system reform. Policy¬ 
makers’ priorities stem directly from the very different position they occupy, at some 
distance from the everyday interactions and social processes that make up teaching 
and learning within particular settings. Instead their eyes are firmly fixed on 
responsibilities and powers that belong in the political domain. Those researchers who 
have worked most closely with policy-makers and then written about the experience 
have often reflected on the difficulties this creates as political priorities change and in 
turn re-shape the research agenda (Luke, 2005; Stannard & Huxford, 2007; Levin, 
2003). Assent in all quarters to the general call to improve teaching and learning often 
blurs the very different interests and motivations that lead such different fields to 
converge on what seems to be the same territory. One important role for the research 
community is to separate out the different logics at work. 
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Ethnographic research tools are well suited to explore this complex terrain. They have 
been variously mobilised to interrogate the different logics that underpin the domains 
of policy, research and practice and how they interact. Using participant observation 
and/or open-ended interviews such studies often focus on close analysis of specific 
social interactions at the level of the classroom. This makes it possible to explore how 
specific policy directives translate into practice, whether and how teachers’ and 
learners’ interactions change in this light, and the intended and unintended 
consequences that may flow from the interaction between the “new” of the policy and 
the “given” of the classroom. Often this kind of enquiry sets out to challenge the reach 
of policy by demonstrating how it is changed as it is enacted through the actions and 
agency of participants on the ground. 

Methodologically ethnographic perspectives assume that people’s actions and beliefs 
are tied to and generated by the specific social contexts in which they act and their 
membership of the networks or communities to which those spaces belong. The 
context makes the culture and the culture shapes the context, in an on-going process 
of negotiation between the two in which regularities emerge, and may become 
relatively fixed. This produces the logic to a given field and the means to act 
successfully within it. Retrieving such regularities in “the rules or norms that 
individuals within a society, community or group have to know, produce, predict, 
interpret and evaluate in a given setting...in order to participate in socially and 
culturally appropriate ways” (Green & Bloome, 1997, p. 186) in respect of 
marginalised or less powerful groups becomes the central point of the analysis. Yet 
such local cultures will also be subject to challenge and renegotiation as one 
community’s norms inevitably come into contact with another’s. The rights to define 
whose norms will prevail in any given setting may well be distributed unequally 
(Michaels, 1986). The use of ethnography in education has often highlighted such 
inequalities by investigating contrasts between pupils’ home literacies and the school 
literacy practices they encounter in the classroom (Freebody et al., 1995; Bloome, 
1992; Heath, 1983). 

Where policy-making is concerned, such inequalities are often structural features of 
policy itself. Policy defines the boundaries to the problem that practice should address 
and the methods and means to be adopted to achieve change. Policy also defines how 
the outcomes of any intervention will be judged, often using repeated and systematic 
collection of perfonnance data to track the progress being made in achieving policy 
goals. There are deliberate asymmetries in agenda-setting and in fonns of 
accountability. Linear-rational models of policy-making regard such inequalities as a 
virtue (Nutley, Davies & Walter, 2007). Provided the choice of policy is steered by 
the best research evidence, then it could and should over-rule the specific 
characteristics of local contexts. This is how improvement will happen. The quality of 
the evidence entitles the policy-makers to be confident that the policy will “work” in 
achieving the desired outcomes. As the policy is implemented, success in changing 
practice rests therefore with policy “fidelity” - the understanding that the terms and 
conditions that make the policy effective will not be diluted or altered by those who 
implement it. Yet adhering to this approach is not without its difficulties, particularly 
when policies seek to intervene and alter complex patterns of social interaction of the 
kind that characterise teaching and learning (Cuban, 1993; Datnow, Hubbard & 
Mehan, 2003). 
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By contrast, constructivist models of policy-making recognise that the relations 
between fields are not straightforward, and that effective policy implementation may 
precisely require adaptation of policy content and conduct to circumstances that the 
policy has not foreseen (Datnow et ah, 2003; Fullan, 1993; Earl et ah, 2003). Those 
championing this approach argue that good use of research evidence in policy 

involves two way rather than linear flows of knowledge in which researchers and 
research users each bring their own experience, values and understanding to bear in 
interpreting research and its meaning for local contexts. Using research emerges as a 
social and collective....process that takes place through intricate interaction. (Nutley 
et al., 2007, pp. 116, 305.) 

From this perspective policy implementation is part of an on-going dialogue between 
interested parties, which has the capacity to modify policy itself as well as local 
actions. These two different approaches create very different vantage points from 
which to consider the interaction between policy and practice. Perhaps surprisingly, 
policy-makers have themselves drawn on the second as well as the first approach 
(Levin, 2010; Bransford, Stipek, Vye, Gomez & Lam, 2009). In stepping into such 
contested terrain, characterised by competing frames of reference that shape literacy 
policy implementation in different ways, ethnographic methods can usefully explore 
how each set of actors regard the specific case, disentangling the logic that constructs 
each point of view. This also means suspending judgement on whose approach is to 
be preferred, and instead setting out to understand what the consequences are of 
deploying different logics in competing fields (Schwartz & Kardos, 2009). 


ETHNOGRAPHIES OF LITERACY POLICY: REFRAMING POLICY TIME 
AND SPACE 

Ethnographic approaches to studying literacy in context have been most fully 
developed in what is sometimes known as the literacy as social practice perspective, 
or the New Literacy Studies (Street, 2003; Barton & Hamilton, 1998; Moss, 2007). 
In this research tradition, the predominant unit of analysis is the literacy event, most 
commonly defined as any occasion where a piece of written text plays an integral part 
in what is going on (Heath, 1983; Barton, 1994). Documenting and analysing a given 
event situates the text involved in relation to its social context of use. What 
participants say and do offers a window onto their shared assumptions about what this 
particular text stands for. Analytically, this gives precedence to those ways of doing 
reading and writing that may be culturally specific to the participants in the immediate 
setting. Steering by the event is intended to precisely bring these aspects of literacy 
practice to light. 

However, in their article, “Limits of the local: Expanding perspectives on literacy as a 
social practice”, Brandt and Clinton argue that focusing too closely on the 
“ethnographically visible” risks according too much agency to those observed 
interacting in the present moment - what they call the here and now of the event - and 
assigning too little power to elements outside the immediate context which 
nevertheless exert a structuring effect (Brandt & Clinton, 2002, p. 343). 

Literate practices are not typically invented by their practitioners. Nor are they 

independently chosen or sustained by them. Literacy in use more often than not 
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serves multiple interests, incorporating individual agents and their locales into larger 
enterprises that play out away from the immediate scene. Further, literate practices 
depend on powerful and consolidating technologies - technologies that are 
themselves susceptible to sometimes abrupt transformations that can destabilise the 
functions, uses, values and meanings of literacy anywhere. (Brandt and Clinton, 
2002, p. 338) 

By consolidating technologies they partly mean artefacts present in the event, which 
they describe as “active mediators - imbuing, resisting, recrafting" the event and its 
meaning through their presence (p. 346). 

In literacy events shaped by literacy policy such artefacts might include the policy 
texts that set the new course of action in motion by their immediate presence. But 
such events might also respond to or be partially shaped by those policy documents 
that never reach the classroom directly but do their work elsewhere within the policy 
sphere, establishing political buy-in for the policy-makers’ direction of travel amongst 
other policy-makers. The publication of Excellence and Enjoyment (DfES, 2003) 
would be an example in the context of the policy development of the National 
Literacy Strategy in England. (See Alexander, 2004). Such documents may 
nevertheless set the tone, the backdrop to the particular literacy event, in important 
ways without being directly invoked there. They may equally be influenced by 
pedagogic resources, whose travel into the classroom is directly sanctioned or 
expedited by policy, even though they have not been fashioned within the policy 
domain. Or literacy events may become saturated by new modes of accountability that 
effectively make what happens in the here and now take its meaning from what will 
happen later, when the pupils’ perfonnance is ultimately judged against a standardised 
test score, or the teacher’s perfonnance is reviewed. In myriad different ways, this 
literacy event, in which teachers and pupils interact to get something done in the here 
and now is therefore profoundly shaped by other times and other places. It cannot be 
considered as a self-sufficient moment, created by the particular participants as a 
direct and unmediated reflection of their own values and beliefs. 

Policy sociology has been particularly interested in exploring the fractured and multi¬ 
layered effect achieved by the simultaneous invocation of times and spaces that are 
elsewhere in the here and now. Most commonly this has been done by tracking the 
evolution of the particular policy over time and in the many different contexts it 
travels through. Shifts in policy-time and policy-space are important (Taylor et al., 
1997). They open up points at which the policy will be recontextualised - re- 
appropriated in ways that simultaneously re-order the particular field the policy 
moves into and the relations between the various actors involved at whatever level 
(Bernstein, 1996). 

The successive realignments between policy and the other spheres of action it touches 
have been variously described. Some focus on establishing timelines to the different 
stages the policy goes through. Levin (2001) suggests: origins, adoption, 
implementation and outcomes. Others prioritise the successive contexts in which 
policies impact: national; district; school, classroom (Taylor, Rizvi, Lingard & Henry, 
1997). In these ways it becomes possible to identify how policy work changes at 
different points in the policy lifecycle. Earl et al. (2003), drawing on Wylie (2002), 
suggest development, evolution and transformation as the successive stages through 
which policies mature. They see the latter two as inevitable consequences of policy 
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interacting with unanticipated features of the local settings in which it is implemented. 
In some cases they argue that policy transfonnation will lead to the policy’s demise. 

In some respects these categories articulate with policy-makers’ own attempts to 
makes sense of policy development. These separate out the kinds of information 
necessary to steer decisions about what kinds of policies are required, through the 
point of policy design, to the attempts to effectively implement the policy and the 
points of review and evaluation afterwards in which decisions can be taken about 
what to do next. In the New Labour period of office in the UK, this was sometimes 
referred to as the ROAMEF framework, in which policy-makers would be expected to 
adopt different courses of action as they established a Rationale for the policy, its 
Objectives, spelt out in specific and measurable terms, Appraisal of the various means 
by which these objectives could be met, Monitoring of the policy in action, 
Evaluation of what it had achieved, and feedback on its success to inform subsequent 
decision making (.Evaluation and Appraisal Guidance, DfES, 2004). These are 
attempts both to rationalise the decision-making process and exercise control over 
likely outcomes. In many respects they also represent the recontextualisation of 
research tools, associated with action research, into the policy domain (Moss, 2009). 

By contrast, over time, policy sociology has built up a different picture of how these 
sequenced interactions work. This turns out to be a much less tidy process than the 
original models predicted, in part because a single policy rarely has unlimited time or 
space in which to run its course. Rather, individual policies provide a single thread 
linking one context to another when the relation between whole fields may be shifting 
in other ways. Stephen Ball’s work captures some of these multi-dimensional 
processes precisely by setting out to track other kinds of policy actors and networks 
that are also reaching into the policy sphere. His most recent work has moved away 
from identifying the trajectory that individual policies may follow as they cycle 
through contexts of influence, text production and practice to instead describing more 
complex re-orderings of the policy scape, in which the flow of knowledge, finance, 
and networks that now generate policy change produce disparate effects and 
temporary conjunctures in a disordered world. Thus, Ball comments in a review of 
Academy schools in England, a new form of school provision akin to charter schools 
in the USA: 

New policy communities bring new kinds of actors into the policy process, validate 
new policy discourses - discourses flow through them - and enable new forms of 
policy influence and enactment....These new forces are able to colonise the spaces 
opened up by the critique of existing state organisations, actions and actors....All in 
all it replaces hierarchy with heterarchy ,...bureaucracy and administrative structures 
and relationships with a system of organisation replete with overlap, multiplicity, 
mixed ascendancy and/or divergent-but-coexistent patterns of relation. (Ball, 2009, 

p.100) 

All of this hectic activity argues against keeping a narrow lens on whether and how 
far a particular policy content or strategy delivers new forms of pedagogic practice to 
particular settings and changes pupil outcomes in its wake. Important as it may be to 
understand how policy impacts on practice, this is not the only or even necessarily the 
key question. Rather, refraining how ethnography looks at the interaction of time and 
space in the here and now linking this moment to other times, other settings, other 
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actors and other interests can help explain what sustains or undermines change in the 
classroom 

In their paper, Brandt and Clinton highlight the need to account for what they call the 
“transcontextual aspects of literacy”, those facets of literacy practice that “regularly 
[arrive] from other places - infiltrating, disjointing, and displacing local life” (2002, 
p. 343). In relation to literacy policy this raises a number of questions about how 
policy itself gets into the room. What are the means it uses to re-shape the local 
sphere? And in that case where should we look to see its effects? One route is to 
continue to take the particular event, a moment in teaching and learning that 
instantiates the interaction between literacy policy and pedagogic practice at the local 
level, and track back to those other contexts and spaces that detennine this encounter. 
Micro-analysis of such moments can facilitate more nuanced accounts of policy 
effects, its tension points and contradictions, and the intended and unintended 
consequences that flow through policy implementation. This might still entail 
describing how the new of policy is re-appropriated into older grammars of place or 
time and the individual values and beliefs that sustain them. A case study conducted 
in England by Adam Lefstein (2008a) provides a good example. 

Lefstein set out to explore the use of exemplar teaching materials produced as part of 
the National Literacy Strategy in 2002 with the final year of primary school (Year 6) 
in mind. The moment he captured stemmed from the lesson of an experienced teacher 
working in a low performing school with the age group whose test scores mattered 
most in detennining the school’s ranking. Lefstein’s interest was in how these 
materials got taken up. The model materials were designed to encourage and develop 
“higher order reading skills” that “probe beyond the literal” through developing 
teachers’ skills in using open questions to structure pupil talk (Lefstein, 2008a, p. 
713). They set out a sequence which shifted control over the teaching and learning 
activities that accompanied a short story from the teacher to the pupils, working 
independently or in pairs, back to the teacher again. The materials were therefore 
intended to reapportion time and space in the classroom so that pupils could apply 
their “higher order reading skills” to a particular set of issues the story raised. The 
materials saw some value in pupils talking about the text in a relatively open way, 
using the text to sustain their enquiries and shape the discussion. The interaction that 
Lefstein documented showed many of the open-ended questions suggested in the 
materials being used. But by running the activity from within the whole class space, 
the teacher compressed the time pupils had to investigate and speculate on their own. 
Lefstein comments: 

Rather than releasing the pupils to work independently for 20 consecutive minutes, 
Mr Thompson punctuated their independent work with frequent interventions, 
collecting answers to one task and assigning a new one. The longest period of 
uninterrupted pupil work is 5.5 minutes. On the second day Mr Thompson skipped 
the independent task altogether, instead continuing whole class teaching until the end 
of the lesson. (2008a, p. 717) 

Through micro-analysis of particular exchanges, Lefstein argues that “The 
suppression of potentially open questions was the joint accomplishment of both Mr 
Thompson and his pupils” (p. 720) as they converged on another model of spoken 
interaction with the respective roles it allots to both pupils and teachers. Each were 
playing their own part in this realignment of interactional genres. The result was that 
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“potentially rich and challenging inquiry is broken down into a series of closed 
questions leading to a rather banal conclusion” (p. 723) and “the prescribed tasks and 
questions were assimilated into the class’s habitual ways of interacting....Macro level 
goals regarding the use of open questions were often subverted by micro-level 
patterns” (p. 724). 

Lefstein sees this episode as a problem for policy and a problem for pedagogy. He 
sets his analysis of policy in the context of the prescriptive approach NLS took to 
school reform. In its earliest stages, teachers were left little room to negotiate with the 
policy’s principles. Rather they were expected to follow exactly its prescriptions and 
would be explicitly judged on whether they had done so through a planning regime 
run as high-stakes auditing. One might add that the release of these materials designed 
to support more exploratory and open-ended readings of a given text also happened at 
a moment in the policy cycle when early performance gains had slowed, and there 
was particular anxiety about the number of pupils reaching higher-level standards. 
However, the materials themselves arrived without such a sub-text being made 
explicit, to supply a missing gap in the programme identified elsewhere (Moss, 2009). 
By contrast, the engagement with policy that Lefstein records within the school was 
acute awareness of its poor record in raising results, despite high levels of compliance 
with what the Strategy asked of them. This disjuncture unsettled staff. 

In terms of this teacher’s engagement with the materials, Lefstein treats this as neither 
a case of misunderstanding nor resistance. On the contrary, he is at some pains to 
show that this teacher did not regard the text as closed but valued different 
interpretations of the text and looked for ways to elicit and sustain them. He suggests 
that the real difficulty in the classroom was the enduring stability that the interactional 
genres associated with whole-class settings afford and the lack of appropriate 
professional support for re-working these. Lefstein comments: 

The professional development opportunities provided by the NLS were based largely 
on a demonstration and imitation model, in which the teachers observed live or video 
demonstrations, which they were then expected to emulate. None of the complexities 
or problematic aspects of such teaching were explored. (2008a, p. 727) 

There are other ways of deploying ethnographic research tools to map policy space 
and time. Colin Mills (2011), also writing about English primary schools, has studied 
the interactions between external consultants, brought into schools via different routes 
to improve school perfonnance, and the staff they set out to help. The study does not 
so much focus on individual literacy events as use participant observation and open- 
ended interviews to tease apart the very different contexts consultants and teachers 
gear towards in their respective roles. Often in his study, the starting point for the 
discussion is the perfonnance data generated at the level of the school. This is at once 
impersonal and overly precise in the common object it constructs “between two 
worlds”. Impersonal in so far as it deals in generalities, not the specific individuals, 
their purposes and micro-patterns of social interactions that generate the data; but also 
quite precise in terms of the focus for future action. He quotes a consultant’s input to 
a staff meeting in a school in “special measures” - a designation that leads to a 
programme of intensive external support designed to raise standards. 

The main problem you can see from the online data, is in boys’ achievement...and we 
can further analyse the figures to see that it is boys’ writing that is bringing you down 
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in the local authority rankings, and in the national figures. You are getting 57% to 
Level 4 at Year 6 - that’s way below the authority benchmark....75% and way below 
the actual figure for the authority which is 80%. That’s also the national figure we are 
reaching. When you have a close look you can see it is the boys who are holding you 
back. (Mills, 2011, p. 106) 

Mills comments that the consultants are primarily positioned by their accountability to 
government and local authority policy not the school. They home in on rankings and 
figures that place this school in relation to others - the tangible signs that will also 
demonstrate when they have done their job. Their gaze is thus firmly fixed on the 
outcome sought, on what this school should be if it would only measure up. By 
contrast the teachers he interviewed articulate their response to the perfonnance data 
differently, by recontextualising objective data into personal relationships with pupils 
and other staff, and by recalling longer histories to the school and the ways of 
working that have evolved there over time. They put back in the contexts the data 
strips out. He treats this orientation as deeply tied to their sense of professional worth 
and their conceptions of teaching and learning. The struggle here is not so much over 
different perspectives on teaching and learning or the value of different approaches 
per se; rather it is how each side is invested in solving the “problem” that lies at the 
heart of the interaction between these two different groups. 

Finally, it is also possible to pursue literacy policy into other arenas that are further 
away from the site of schools and the immediate contexts in which teaching and 
learning take place. Two further examples follow. In The politics of literacy in the 
context of large-scale reform (Moss, 2009), I drew on interview data I had collected 
from a range of policy-makers involved in developing the National Literacy Strategy 
in England between 1996 and 2004. They included politicians, civil servants, Heads 
of Agencies with responsibilities that brought them closer to or put them further from 
direct management of the Strategy, as well as those directly involved in both its policy 
design and implementation. The interviews were semi-structured and designed to 
elicit interviewees’ views of how the policy had evolved, key events, documents and 
turning points they considered had shaped its evolution and what they had learnt 
about policy or research from observing or participating in the process. This revealed 
how far the trajectory of the policy was shaped by structural features of the policy 
environment itself - the annual decision-making cycle in policy-making; the courses 
of action determined by the patterns of government finance; and the uncertain 
trajectory to any such initiative that plays out in the political sphere. Indeed major 
shifts in the direction the policy took were directly linked to political weaknesses 
expressed in public debate at the centre rather than the strength of the research 
evidence about the policy’s relative success or sustainability (See also Moss & 
Huxford, 2007). Public events in the policy sphere often took precedence over what 
was really happening in schools. One conclusion I drew from the analysis was that the 
difficulty policy-makers found in fixing the results from schooling to their satisfaction 
over the longer term ultimately led them to decide to give the problem away. In the 
English context this is leading to system fragmentation and an increasing desire to 
invite private operators in to pick up the mess of broken promises the state has left 
behind. 

By contrast, the interplay between political intent, public debate and research 
knowledge that lies at the heart of so many of these issues finds an alternative means 
of exploration in Lefstein (2008b). In this final example, Lefstein analysed three 
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episodes of Newsnight, broadcast in 2005. One of the most influential news and 
current affairs programmes on the BBC, Newsnight had decided to try to explain why 
there was then so much interest in introducing systematic phonics instruction into the 
early years as a potential answer to the perceived failings of the NLS. They did so by 
documenting the introduction of a commercial synthetics phonics programme to a 
school with previously poor literacy standards. In practice this led to a sequence of 
reports which packaged pre-recorded material collected in the school with a live 
studio debate. Unusually, Lefstein presented his analysis to the participants in the 
studio debate as well as the report’s producer at Newsnight ahead of publication. The 
article explores how the structuring of the news report cast speakers as protagonists 
for and against phonics and phonics itself as an easy answer to often complex literacy 
problems - views which did not fully fit with any of the experts they asked to discuss 
their experiment. Yet “expert knowledge” itself became problematic in this context, 
given the limited space in which it was possible to speak, and the terms in which 
contributions were allowed. Lefstein wonders if part of the difficulty here is the extent 
to which the Newsnight report was relying on the genre of the makeover to do the 
work of telling a complex story in simple terms. He concludes “proponents of 
evidence-based policy...might take comfort in the fact that educational research 
received considerable attention on Newsnight. However, although “research” was 
elevated, the researcher who assumed the role of scientific gatekeeper was 
marginalised. Paradoxically the case study suggests that “what works” in the public 
sphere may be a less “scientific” approach” (p. 1141). 


ENGAGING WITH POLICY: THE ROLE OF EVIDENCE AND THE 
RESPONSIBILITIES OF RESEARCHERS 

Often in the academic literature on literacy policy, what seems to matter most is who 
controls the policy content. This can be treated as a re-run of long-standing disputes 
within the research field: about pedagogic method - whole books versus phonics; or 
about research methodologies - quantitative versus qualitative approaches. On the 
contrary, what I am arguing here is that policy does not provide a neutral space in 
which any of these methodologies can impose its own tenns on an already congested 
social space. Policy logics are not the same as research. They have developed in very 
different social contexts. Policy recontextualises into its own image knowledges and 
approaches whose logic arises elsewhere, embedded in very different kinds of social 
relations. In setting such knowledge to work policy sutures space and time together in 
new ways. 

Research does indeed have responsibilities to practice and to making literacy learning 
and teaching function as successfully as they can. It has far less responsibility to the 
policy field, in terms of either guaranteeing or underwriting the promises that 
politicians make on their own terms and detennining whether they can really be 
honoured to the letter. Part of the difficulty at the moment may simply be that 
politicians promise too much, provide too little time or security in policy delivery and 
probably expect to pay too little. 

Where research does have a clear and pressing role in policy is in assessing as 
objectively as it can the diverse impacts that policy has. I would argue this is a 
democratic duty. This can be done using a wide variety of approaches. Quantitative 
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analysis of large-scale datasets may be at least as important as qualitative and small- 
scale studies of particular cases at the level of the individual school, classroom or 
community. Both kinds of methodologies have a place. But tensions between them 
should not expect to get resolved according to which ones get taken up by policy¬ 
makers and co-opted into their sphere. 
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